Skip to content

Prbatero/feat/data publishing pc - #123

Open
Anthony Cintron (prbatero) wants to merge 17 commits into
mainfrom
prbatero/feat/data-publishing-pc
Open

Prbatero/feat/data publishing pc#123
Anthony Cintron (prbatero) wants to merge 17 commits into
mainfrom
prbatero/feat/data-publishing-pc

Conversation

@prbatero

Copy link
Copy Markdown
Contributor

Description

This pull request adds a new publishing queue processing system to the api/hastefuncqueues Azure Functions app, introduces new dependencies for publishing support, and expands the artifact storage interface for improved cloud operations. It also includes new tests for the publishing handlers and updates configuration for local development and deployment.

Publishing queue and handlers:

  • Added three new Azure Functions in function_app.py to process dataset publishing operations: GetPublishDatasetQueueMessage, GetPublishDatasetPoisonQueueMessage, and ReconcilePublishingOperations, which handle publishing, poison queue messages, and reconciliation of stale operations, respectively.
  • Introduced the PublishingProcessor and PublishQueueMessage imports to support the new publishing workflow. [1] [2]

Dependency and configuration updates:

  • Added azure-planetarycomputer and pystac[validation] to requirements.txt for publishing support, and updated azure-core to 1.39.0. [1] [2]
  • Added a planetary-computer optional dependency group in pyproject.toml for the publishing target, including geospatial libraries.
  • Updated docker-compose.yml to include publishing-related environment variables for local development and testing. [1] [2]

Artifact storage enhancements:

  • Extended the AbstractArtifactStorage interface with new abstract methods for resolving paths, copying, deleting, checking existence, and getting metadata for artifacts, supporting more robust cloud operations.
  • Improved AzureBlobArtifactStorage initialization for thread-safe container creation, added identity-based client, and made store_artifact method more robust to None values. [1] [2] [3] [4] [5]

Testing:

  • Added a new test module test_publishing_handlers.py with comprehensive tests for the publishing queue handlers, covering normal operation, error cases, and poison queue handling.

Fixes #

Type of change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that changes existing behavior)
  • Documentation update
  • Infrastructure / CI change

Checklist

  • I have read CONTRIBUTING.md
  • My changes follow the project's coding standards (PEP 8 for Python, ESLint rules for JS/TS)
  • I have added or updated tests that cover my changes
  • Python tests pass locally (cd hastelib && hatch run test:pytest) and the UI lints clean (cd ui && npm run lint)
  • I have updated the relevant documentation (README, docs/, inline comments)
  • I have added an entry to CHANGELOG.md if this is a user-facing change

Testing

Additional context

Anthony Cintron Roman and others added 8 commits August 11, 2026 13:37
…lease

Bucket A (additive) from the -sol audit split, Phase 1 (Local only; no PC):
- core/publishing/: base ABC + registry (PC lazy-loaded, disabled by default),
  repository (per-dataset docs + optimistic concurrency), source (eligibility +
  artifact resolution), lease (blob-lease coordinator), local_provider (immutable
  copy to published/{datasetId}/).
- models/publishing.py; processors/publishing.py + assessment.py.
- Additive infra: config publishing block + PUBLISHED_DATASET type + publish
  queue; data-layer load_bounded/load_page/update_locked; artifact-storage
  copy/scoped-download/etag/size/exists/delete_prefix; blob.py max_bytes.
- hastefuncqueues publish trigger.
- Phase-1 unit tests (registry/repository/source/local_provider/lease): 45 pass.

Excludes Bucket B: reverted the high-risk load_all/load_all_from_partition
rewrite to main's versions; endpoint hardening + DTOs left for a separate
-hardening PR. See spec/features/data-publishing/sol-classification.md.
Bucket A only. Adds to api/hastefuncapi/function_app.py: publishing imports,
active-caller/authorization helpers, the DeleteProject published-dataset guard,
and 7 publishing routes (GetPublishingProviders, GetPublishDatasetOptions,
GetPublishedDatasets, GetPublishedDataset, PutPublishDatasetQueueMessage,
PutRetryPublishedDatasetQueueMessage, DeletePublishedDataset). All existing
endpoints reverted to main's versions — the Bucket-B endpoint hardening (IDOR
checks, strict *Request DTOs, server-managed-field enforcement, assessment size
caps) is deferred to a separate -hardening PR.

Tests: 19 publishing route tests + 6 queue-handler tests pass. Existing core
suite: 191 pass (3 failures are pre-existing on main — clipBbox mocks).
Bucket A UI, gated behind the publishingEnabled flag:
- New: PublishedDatasets section, PublishedDatasetRow, PublishDatasetModal
  (name/description prefill, asset checklist, provider-driven target dropdown),
  util/publishing.js, util/assessmentSummary.js (extracted from AssessmentReportModal).
- Additive wiring: App/AppContext (publishingEnabled + provider fetch),
  AppBody route + AppSidebar nav, "Publish dataset..." in ModelResultsButton +
  EmbeddingModelRow, Database icon.
- api.js: apiPut/apiDelete now accept any 2xx (async 202 queue responses) —
  minimal additive change; the shared-helper ApiError rewrite is NOT taken
  (deferred to -hardening). Existing 200 callers unaffected.

Validated: full `vite build` compiles cleanly (all imports resolve). Vitest
unit tests + config are preserved on -sol/archive and land in a follow-up
UI-test-infra slice (needs vitest devDeps + package-lock regen).
UI (Published Datasets section):
- Clickable Project/Layer cell -> opens the source project with that layer
  expanded (/project/:projectId/:imageLayerId).
- Labeled "Actions" button (replacing the bare "...") and an "Actions" column
  header; keep the menu (download / open in Explorer / retry / unpublish).
- New "View details" action -> modal with the generated description, project/
  layer, model, target, status, publisher, headline assessment stats, and the
  published assets (kind + size).
- "Published by" now mirrors the project Creator display: the publisher's
  login/email via limitTextLength, not the opaque object id.
- Desktop column widths moved into a @media (min-width: 993px) rule scoped to
  .pgrid-page--published-datasets so the mobile card layout (inherited from
  .pgrid-page--model-catalog) is preserved; font URLs forced to https (CSP).

Backend:
- Capture the publisher's display name (client-principal userDetails) at publish
  time as PublishedDataset.publishedByName, so the UI can show a human name while
  the ownership check still uses the object id.
Add the data-publishing feature flag and queue name to the api/queues
Function App settings so the Local target runs in Azure:

- functions.bicep: PUBLISH_QUEUE_NAME ('publish-queue') and
  PUBLISHING_ENABLED (from a new publishingEnabled param) in appConfigSettings.
- main.bicep / main.bicepparam: publishingEnabled param, default off,
  sourced from HASTE_PUBLISHING_ENABLED.

The publish-queue and the publishing-locks blob container are auto-created at
runtime, and APIM operations are auto-synced from the deployed function app by
the postdeploy hook, so no queue/container/APIM resources are needed here.
Other Local knobs (PUBLISH_MAX_TOTAL_BYTES, PUBLISHED_DOWNLOAD_SAS_MINUTES,
PUBLISHING_LOCK_CONTAINER) use code defaults.

Note: infra/main.json (compiled ARM artifact) is intentionally not regenerated
here to avoid a bicep CLI-version reformat; regenerate it with the repo's
pinned bicep version.
Add the publishing feature flag + queue/lock settings to the api and queues
services so `docker compose up` exercises Local publishing against Azurite
(the publish-queue and publishing-locks container auto-create at runtime).
PC_PROVIDER_ENABLED stays false — no local GeoCatalog.
Drop the accidental "install" and "npm" entries from the UI dependencies (and
their lockfile trees). Neither is imported anywhere; the npm CLI as a runtime
dependency is what caused local image builds to fetch an npm tarball. Mirrors
the earlier cleanup in #64 before they regressed back in.
… deploy paths

The Config drift check failed with:

  Dead application settings (set but never read):
    PUBLISHING_ENABLED  (set by infra/modules/functions.bicep, read by no code)

That was a false positive. PUBLISHING_ENABLED is read in hastegeo.core.config
via _get_bool_env("PUBLISHING_ENABLED", False), but check_env_drift.py's AST
scanner only recognized os.getenv, os.environ.get, and os.environ[...]. The new
typed wrappers _get_bool_env and _get_bounded_int_env were invisible to it, so
no reader was found and the setting looked dead.

The blind spot covered five variables, not one: PC_PROVIDER_ENABLED,
PUBLISH_MAX_TOTAL_BYTES, PUBLISHED_DOWNLOAD_SAS_MINUTES and PC_VERIFY_ATTEMPTS
are read through the same helpers and were equally unseen. They simply did not
error because no deploy path emits them yet.

Teach the scanner to treat _get_*_env(NAME, ...) calls as reads. The wrappers
supply their own default in the signature, so such a read counts as optional
even when the call site passes no default.

Separately, fix the real drift the check could not see: PUBLISHING_ENABLED was
emitted by functions.bicep but not by deploy_apps.sh, so environments deployed
through the GitHub Actions path would never receive it and would silently fall
back to false while Bicep-provisioned environments got the flag. Emit it from
deploy_apps.sh too, defaulting off to match the Bicep param, and wire it
through deploy-apps.yml as a GitHub Environment variable (non-sensitive feature
flag, same treatment as VITE_SHOW_FOOTER).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds end-to-end dataset publishing through local storage and Planetary Computer, including queue processing, persistence, infrastructure configuration, UI management, and tests.

Changes:

  • Introduces publishing models, providers, storage operations, queue handlers, and reconciliation.
  • Adds publishing dialogs, catalog browsing, downloads, retries, and unpublishing.
  • Extends infrastructure configuration and backend test coverage.

Reviewed changes

Copilot reviewed 65 out of 68 changed files in this pull request and generated 11 comments.

Show a summary per file
File Description
ui/src/util/publishing.js Publishing status and artifact helpers.
ui/src/util/icons.jsx Adds database icon support.
ui/src/util/assessmentSummary.js Extracts assessment summary formatting.
ui/src/util/api.js Supports additional successful HTTP statuses.
ui/src/Components/PublishedDatasets.jsx Adds published-dataset catalog.
ui/src/Components/PublishedDatasetRow.jsx Adds dataset details and actions.
ui/src/Components/PublishDatasetModal.jsx Adds publishing form.
ui/src/Components/ProjectManagement/ModelResultsButton.jsx Adds publish action for model results.
ui/src/Components/ProjectManagement/EmbeddingModelRow.jsx Adds publish action for embeddings.
ui/src/Components/BuildingValidation/AssessmentReportModal.jsx Reuses summary utility.
ui/src/Components/AppSidebar.jsx Adds publishing navigation.
ui/src/Components/AppBody.jsx Registers publishing route.
ui/src/assets/css/style.css Styles publishing table and font URLs.
ui/src/AppContext.jsx Adds publishing state.
ui/src/App.jsx Loads publishing capabilities.
ui/package.json Removes unused npm dependencies.
infra/modules/roles.bicep Adds optional ingestion storage role.
infra/modules/functions.bicep Adds publishing app settings.
infra/main.bicepparam Maps publishing environment parameters.
infra/main.bicep Wires publishing infrastructure parameters.
hastelib/tests/core/publishing/test_source.py Tests source validation.
hastelib/tests/core/publishing/test_repository.py Tests persistence and concurrency.
hastelib/tests/core/publishing/test_registry.py Tests provider registration.
hastelib/tests/core/publishing/test_planetary_computer_transport.py Tests GeoCatalog transport.
hastelib/tests/core/publishing/test_local_provider.py Tests local publishing.
hastelib/tests/core/publishing/test_lease.py Tests blob lease coordination.
hastelib/src/hastegeo/core/utils/blob.py Adds bounded streaming downloads.
hastelib/src/hastegeo/core/publishing/source.py Resolves verified source artifacts.
hastelib/src/hastegeo/core/publishing/schemas/projection-v2.0.0.json Vendors projection schema.
hastelib/src/hastegeo/core/publishing/schemas/item-assets-v1.0.0.json Vendors item-assets schema.
hastelib/src/hastegeo/core/publishing/repository.py Adds publishing persistence.
hastelib/src/hastegeo/core/publishing/registry.py Adds provider registry.
hastelib/src/hastegeo/core/publishing/planetary_computer_transport.py Adds GeoCatalog adapter.
hastelib/src/hastegeo/core/publishing/local_provider.py Adds local provider.
hastelib/src/hastegeo/core/publishing/lease.py Adds distributed operation locks.
hastelib/src/hastegeo/core/publishing/geocatalog_client.py Adds authenticated REST client.
hastelib/src/hastegeo/core/publishing/base.py Defines provider contract.
hastelib/src/hastegeo/core/publishing/__init__.py Exports publishing primitives.
hastelib/src/hastegeo/core/processors/metadata.py Adds bounded and paged reads.
hastelib/src/hastegeo/core/processors/assessment.py Adds assessment snapshot processor.
hastelib/src/hastegeo/core/models/publishing.py Defines publishing schemas.
hastelib/src/hastegeo/core/data_layer/unified.py Exposes paged storage operations.
hastelib/src/hastegeo/core/data_layer/local_file_system_data_layer.py Adds bounded local reads.
hastelib/src/hastegeo/core/data_layer/azure_postgresql_data_layer.py Adds bounded PostgreSQL reads.
hastelib/src/hastegeo/core/data_layer/azure_data_lake_data_layer.py Adds bounded Data Lake reads.
hastelib/src/hastegeo/core/data_layer/azure_cosmos_db_data_layer.py Adds bounded Cosmos reads.
hastelib/src/hastegeo/core/data_layer/azure_blob_storage_data_layer.py Adds indexed blob pagination.
hastelib/src/hastegeo/core/data_layer/abstract_data_layer.py Extends data-layer interface.
hastelib/src/hastegeo/core/artifact_storage/unified_artifact_storage.py Exposes publishing storage operations.
hastelib/src/hastegeo/core/artifact_storage/local_file_system_artifact_storage.py Implements local artifact operations.
hastelib/src/hastegeo/core/artifact_storage/azure_blob_artifact_storage.py Implements cloud artifact operations.
hastelib/src/hastegeo/core/artifact_storage/abstract_artifact_storage.py Extends artifact storage contract.
hastelib/pyproject.toml Adds optional publishing dependencies.
docker/docker-compose.yml Enables local publishing configuration.
api/hastefuncqueues/tests/test_publishing_handlers.py Tests publishing queue handlers.
api/hastefuncqueues/tests/__init__.py Initializes queue test package.
api/hastefuncqueues/requirements.txt Updates queue dependencies.
api/hastefuncqueues/function_app.py Registers publishing queue functions.
api/hastefuncapi/tests/__init__.py Initializes API test package.
Files not reviewed (1)
  • ui/package-lock.json: Generated file

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread ui/src/Components/ProjectManagement/EmbeddingModelRow.jsx Outdated
Comment thread hastelib/src/hastegeo/core/publishing/source.py Outdated
Comment thread ui/src/Components/PublishedDatasetRow.jsx
Comment thread ui/src/Components/PublishedDatasets.jsx Outdated
Comment thread hastelib/src/hastegeo/core/publishing/registry.py Outdated
Comment thread hastelib/src/hastegeo/core/publishing/geocatalog_client.py Outdated
Comment thread hastelib/src/hastegeo/core/config.py
Comment thread infra/modules/functions.bicep
Comment thread ui/src/Components/PublishDatasetModal.jsx Outdated
Comment thread api/hastefuncqueues/requirements.txt
@prbatero
Anthony Cintron (prbatero) force-pushed the prbatero/feat/data-publishing-pc branch from e5dc652 to 9a6d1a0 Compare August 12, 2026 12:43
@github-actions

Copy link
Copy Markdown

RC artifacts ready

All branch deployment references use the same RC tag:

  • hastegeo_version: 1.0.28rc3
  • training_image_tag: 1.0.28rc3
  • imageprep_image_tag: 1.0.28rc3
  • wheel: hastegeo-1.0.28rc3-py3-none-any.whl

…s, polling, privacy

Resolve Copilot review findings on the publishing feature:

- Embedding eligibility: source resolver gated on inferenceStatus, but embedding
  models signal completion via `status`. Gate on `status` for modelType
  "embedding", else inferenceStatus — otherwise every embedding publish was
  rejected before the embedding artifact logic ran.
- Publish action on EmbeddingModelRow used inferenceStatus (never set for
  embeddings); use the already-computed isProcessed.
- Owner detection: populate appParams.identityId from the SWA principal object id
  so it matches PublishedDataset.publishedByUser; previously identityId was never
  assigned, so non-admin publishers were never recognized as owners and
  Retry/Unpublish were hidden.
- Publish dialog reported "Publishing started" on a 409 conflict (apiPut returns
  the status rather than throwing); surface the conflict as an error instead.
- Published Datasets polling captured a stale query (page/filters/search/sort);
  drive the interval through a ref to the latest fetch and pause it while the
  search is not ready.
- Persist the publisher email/login rather than the display name
  (PublishedDataset.publishedByName) — display names resolve from Entra at read.
- Remove aria-modal=false from the publish dialog (it is a focus-trapping modal).
- Provider registry: a Planetary Computer target is "configured" once the
  GeoCatalog URL is set; do not also require an ingestion source (needed only for
  private containers), so the public-container mode is configurable.
@prbatero
Anthony Cintron (prbatero) force-pushed the prbatero/feat/data-publishing-pc branch from 9a6d1a0 to 7fca5ab Compare August 12, 2026 13:09
@github-actions

Copy link
Copy Markdown

RC artifacts ready

All branch deployment references use the same RC tag:

  • hastegeo_version: 1.0.28rc4
  • training_image_tag: 1.0.28rc4
  • imageprep_image_tag: 1.0.28rc4
  • wheel: hastegeo-1.0.28rc4-py3-none-any.whl

Publishing adds the queues app's first timer trigger
(ReconcilePublishingOperations), which acquires a host-scoped Singleton lock. A
colliding auto-generated host id (long app name / shared storage / deployment
slots) then surfaces as "Unable to acquire Singleton lock" and NoScriptHost,
leaving the app unhealthy. Document setting a unique AzureFunctionsWebHost__hostId
per app/slot in the rollout operator notes. Applies to Local and PC (the timer
ships with the Local feature).
@prbatero
Anthony Cintron (prbatero) force-pushed the prbatero/feat/data-publishing-pc branch from 154fcd0 to 30f818a Compare August 12, 2026 15:09
@github-actions

Copy link
Copy Markdown

RC artifacts ready

All branch deployment references use the same RC tag:

  • hastegeo_version: 1.0.28rc6
  • training_image_tag: 1.0.28rc6
  • imageprep_image_tag: 1.0.28rc6
  • wheel: hastegeo-1.0.28rc6-py3-none-any.whl

…proxy

Published copies were written to `{hash(projectId)}/published/{datasetId}/<file>`
— a five-segment blob path (data/hash/published/datasetId/file). The UI already
rewrites download URLs through the managed-identity storage proxy
(get-artifacts) via toBrowserStorageUrl, but that proxy only accepts four
segments, so the rewrite returned null and the UI fell back to the raw blob SAS
URL. On the VNet-only storage account (defaultAction: Deny) a direct blob
request from the browser is rejected with AuthorizationFailure.

Flatten the published prefix to `published/{datasetId}/<file>` (three
in-container segments). datasetId is a UUID, so the project-hash level was
redundant for uniqueness. Downloads now match the get-artifacts proxy and are
served on-network via managed identity, like every other HASTE artifact
download.

Note: datasets published before this change keep their old five-segment path and
must be re-published to be downloadable.
@prbatero
Anthony Cintron (prbatero) force-pushed the prbatero/feat/data-publishing-pc branch from 30f818a to 6bdaa62 Compare August 12, 2026 15:38
The section fetched its first page with fetchDatasets(false), so no loading
indicator showed and the page rendered blank (the component returns null while
items === null) until data arrived — noticeable on slower cloud loads. Trigger
the shared full-page loading overlay on the first load (items === null), matching
the Model Catalog pattern; later filter/search/sort/page changes still refetch
silently so the overlay doesn't flash on every interaction.
Anthony Cintron Roman added 4 commits August 12, 2026 15:11
…aining submit)

The initial publishing commit moved the module-level
`from hastegeo.core.processors.train import TrainPreprocessor` import into a
local import inside PutCancelModelQueueMessage, but PutRunModelQueueMessage (and
the retrain path) still reference TrainPreprocessor at module scope. That raised
an unhandled NameError on every training submit, surfaced to the client as a 500
from PutRunModelQueueMessage. Restore the top-level import and drop the now
redundant local one.
Phase 2, stacked on -local. Adds the Planetary Computer publishing target using
a vendored REST GeoCatalog client instead of the azure-planetarycomputer SDK —
offline-verifiable, no proprietary-SDK coupling (per the spec decision).

- geocatalog_client.py: hardened REST client + Entra auth (scope
  .../.default), redirect suppression, per-request timeouts, sanitized errors
  (never embeds server response bodies), api-version 2026-04-15.
- planetary_computer_transport.py: resumable one-request-per-step adapter
  (start/continue create-collection/item, delete-item; get collection/item/
  ingestion-source; asset signing) satisfying the provider's interface. Carries
  over the SSRF operation-URL pinning, endpoint validation, and failed-item
  accounting; FIXES the "Finished" terminal-status bug (was failing good
  ingests). Handles synchronous (201) collection creation vs async (202) items.
- planetary_computer_provider.py + stac.py + schemas/: the resumable bounded-
  step provider and STAC builder (item geometry from the valid-area mask,
  ai4g: stats, one collection per project) reused verbatim from the audited
  implementation; provider gains a branch for sync collection creation.
- config: PC_VERIFY_ATTEMPTS default 5->20 (bound 60). Continue-steps re-enqueue
  at ~30s, so this is a ~10 min default (30 min max) ingestion budget.
- pyproject: planetary-computer extra -> vendored-REST deps (azure-identity,
  requests, pystac[validation], geopandas, pyogrio, pyproj, shapely).

Tests: 97 publishing unit tests (incl. 16 new transport HTTP-fixture tests:
SSRF pinning, Finished, failed-items, sanitization, sync/async, 404/409) + 25
API tests pass. Registry lazy-loads the provider and PC is disabled by default,
so -local is unaffected when PC is off.

Follow-up: end-to-end verification against a live GeoCatalog (confirm the
/sas/sign asset-signing endpoint and sync-vs-async collection creation).
…ion role

Stacked on the Local publishing infra wiring; adds the PC-target configuration
so the Planetary Computer provider runs in Azure:

- functions.bicep: PC_PROVIDER_ENABLED, PC_GEOCATALOG_URL, PC_EXPLORER_URL,
  PC_INGESTION_SOURCE, PC_COLLECTION_PREFIX app settings (new pc* params).
  The api-version and Entra token scope are code constants, not settings; auth
  is managed-identity only.
- main.bicep / main.bicepparam: pc* params, default off/unset, sourced from
  HASTE_PC_* env vars. The GeoCatalog itself is operator-provisioned/external.
- roles.bicep: conditional Storage Blob Data Reader on the HASTE storage
  account for an operator-supplied GeoCatalog ingestion managed identity
  (asset ingestion). Off by default (empty principal id); not needed for public
  containers or the SasToken ingestion path.

The complementary grant (function-app identity on the external GeoCatalog data
plane) is operator-owned and configured out-of-band; the exact GeoCatalog RBAC
role needs verification against a live catalog before wiring.

Note: infra/main.json (compiled ARM) not regenerated here to avoid a bicep
CLI-version reformat; regenerate with the repo's pinned bicep version.
Resolve Copilot review findings on the Planetary Computer target:

- Ingestion source is optional (public-container mode): the provider's info
  isConfigured and _require_configuration no longer require a non-empty
  PC_INGESTION_SOURCE, and start_publish only validates the source when one is
  set. A GeoCatalog URL alone now configures the target — matching the spec, the
  Bicep param, and verified live behavior (public blobs ingest without a source).
- GeoCatalog client no longer appends api-version to absolute operation URLs
  that already carry it, avoiding a duplicate api-version query parameter on
  ingestion-operation polling.
- deploy_apps.sh (GitHub Actions deploy path) now emits the PC_* Function App
  settings (PC_PROVIDER_ENABLED, PC_GEOCATALOG_URL, PC_EXPLORER_URL,
  PC_INGESTION_SOURCE, PC_COLLECTION_PREFIX) to match infra/modules/functions.bicep.
- Remove the unused azure-planetarycomputer dependency from hastefuncqueues
  (the implementation uses the vendored REST client, not the SDK).

Also aligns the provider's PC_VERIFY_ATTEMPTS upper bound (was 20) with the
config's allowed maximum (60), so operator overrides above 20 no longer fail
validation at publish time.
The PC provider's stac.py imports geopandas/pyproj (and reads GeoPackages via
pyogrio, validates STAC via pystac). These are lazy-loaded only when resolving
the Planetary Computer target, so they don't affect host startup — but a PC
publish (queue worker) or a PC-target validate (api create path) would fail at
runtime with ModuleNotFoundError. Add geopandas/pyproj/pyogrio (and pystac to
the api) to both function apps' requirements so PC publishing works end to end.
Local publishing does not import these and is unaffected.
@prbatero
Anthony Cintron (prbatero) force-pushed the prbatero/feat/data-publishing-pc branch from 8a13ae3 to 697d356 Compare August 12, 2026 19:11
@github-actions

Copy link
Copy Markdown

RC artifacts ready

All branch deployment references use the same RC tag:

  • hastegeo_version: 1.0.28rc10
  • training_image_tag: 1.0.28rc10
  • imageprep_image_tag: 1.0.28rc10
  • wheel: hastegeo-1.0.28rc10-py3-none-any.whl

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants